Web Survey Bibliography
Web surveys have been a commonly used method for collecting and analyzing data about social and psychological phenomena in online communities. Nevertheless, as yet, little systematic methodological research has been done about administrating and conducting surveys in these online domains. This is somewhat surprising due to the methodological advances of other research techniques in the field of online community research such as virtual ethnography, data mining and social network analysis and to the fact that internet surveys were among the first techniques used in online community research (Jones 1998, 1999).Quality of web surveys in online communities is commonly affected by two types of survey errors (Andrews et al. 2003): non-coverage and unit nonresponse error. Although both errors can be interrelated, on one hand, the noncoverage error mainly applies to the issue of defining a valid unit of analysis (on theoretical and empirical level). Generally online communities require from individuals a kind of formal registration in order to participate in community activities (e.g., in order to submit a message to a forum thread one must log in with their email to an online community). However, there are also online communities that allow individuals to participate without being necessarily registered (i.e., anonymous users). In such cases, researchers often opt for intercept surveys and/or unrestricted self-selected surveys (e.g., by posting an invitation message to a forum discussion thread or publish a banner on the website) (Fricker 2010) failing to give any chance of sample selection to some users of the analyzed online community. On the other hand, the unit nonresponse error generally pertains to list-based web surveys (Lozar Manfreda et al. 2002). In such cases a sampling frame of online community members is available (e.g., a list of all members of an online community), but due to various reasons some sample units refuse to answer the survey. As in general population surveys also in online community surveys unit nonresponse can lead to bias. For instance, there is some empirical evidence that a larger percentage of high-frequency online community participants takes part in surveys compared to low-frequency participants and that high-frequency participants reply quicker to survey invitations than low-frequency participants (Andrews et al. 2003).The purpose of this paper is to present a research design of a study aiming at evaluating the effect of survey invitation characteristics and response reluctance on unit nonresponse in list-based surveys of web forum participants. On one hand, we are interested in response-inducing techniques (Dillman et al. 2007). Up to now the literature on list-based web surveys has been generally focused on the topic of personalization of email invitations, their subject lines and graphical design. The focus of this study is on the content of email invitations. Namely, we will attempt to gain an insight into how response rates are associated with request for help, authority, and sense of community. The decision to study a combination of these three elements was theoretically informed by research on online communities, showing that exchange of social support, belonging and norms are among the most salient building blocks of their development and sustainability (Preece 2000, Kraut et al. 2012). Considering the leverage-salience theory (Groves et al. 2000) and the fact that the these elements are not appealing for all online community members to the same extent, putting forward different combinations of the three elements in invitations might therefore result in different response rates. In addition, we would like to explore if and how the three response-inducing techniques are associated with the unit nonresponse bias. In order to address these questions an experiment with a full factorial between-subjects design with eight experimental groups was planned to be implemented. On the other hand, the research design was developed to explore whether there are differences between early and late respondents as well as if sending reminders can reduce the potential bias associated with unit non-response error. The advantage of list-based web surveys in web forums is that nonresponse bias can be analyzed with the record-linkage technique (Porter and Whitcomb 2005) since both respondents and nonrespondents to the survey can be linked to database records of registered web forum users (which generally includes information about frequency of participation, number of posted messages, year of registration, etc.). The presentation will end with an open discussion of potential avenues for optimization of the proposed research design.
Workshop Homepage (abstract)
Web survey bibliography (457)
- Handbook of Research Methods in Health Social Sciences; 2017; Liamputtong, P.
- Necessary but Insufficient: Why Measurement Invariance Tests Need Online Probing as a Complementary...; 2017; Meitinger, K.
- Device and Internet Use among Spanish-dominant Hispanics: Implications for Web Survey Design and Testing...; 2017; Trejo, Y. A. G.; Schoua-Glusberg, A.
- Role of online survey tools in creating temporally accurate Environmental Product Declarations (EPD)...; 2017; Ganguly, I.; Bowers, T.; Pierobon, F.; Eastin, I.
- CAQDAS at a Crossroads: Affordances of Technology in an Online Environment; 2017; Silver, C.; Bulloch, L. S.
- Development and Pilot Test of a Mobile Application for Field Data Collection; 2016; Chiappetta, L.; Kerr, M. M.
- A streamlined approach to online linguistic surveys; 2016; Erlewine, M. Y.; Kotek, H.
- The Effects of Vignette Placement on Attitudes Toward Supporting Family Members; 2016; Lau, C. Q., Seltzer, J. A., Bianchi, S. M.
- Using Web Panels to Quantify the Qualitative: The National Center for Health Statistics Research and...; 2016; Scanlon, P. J.
- Bees to Honey or Flies to Manure? How the Usual Subject Recruitment Exacerbates the Shortcomings of...; 2016; Snell, S. A., Hillygus, D. S.
- The Use of a Nonprobability Internet Panel to Monitor Sexual and Reproductive Health in the General...; 2015; Legleye, S; Charrance, G.; Razafindratsima, N.; Bajos, N.; Bohet, A.; Moreau, C.
- GreenBook Research Industry Trends Report; 2015; Murphy, L. (Ed.)
- Does Sequence Matter in Multimode Surveys: Results from an Experiment; 2014; Wagner, J., Arrieta, J., Guyer, H., Ofstedal, M. B.
- The Use of Cognitive Interviewing Methods to Evaluate Mode Effects in Survey Questions; 2014; Gray, M., Blake, M., Campanelli, P.
- Build your own social network laboratory with Social Lab: a tool for research in social media; 2014; Garaizar, P., Reips, U.-D.
- Using Eye Tracking to Evaluate Email Notifications of Surveys and Online Surveys Collecting Address...; 2014; Olmsted, E. L., Nichols, E. M.
- Correlates of Attrition in the German Internet Panel: Drop-Outs and Sleepers; 2014; Blom, A. G., Beissel-Durrant, G.
- Survey Breakoff in Online Panels; 2014; McCutcheon, A. L.
- Inside the Turk Understanding Mechanical Turk as a Participant Pool; 2014; Paolacci, G., Chandler, J.
- Nonresponse and measurement error in an online panel; 2014; Roberts, C., Allum, N., Sturgis, P.
- Estimating the effects of nonresponses in online panels through imputation; 2014; Zhang, W.
- Professional respondents in nonprobability online panels; 2014; Hillygus, D. S., Jackson, N. M., Young, M.
- Informing panel members about study results; 2014; Scherpenzeel, A., Toepoel, V.
- Determinants of the starting rate and the completion rate in online panel studies; 2014; Goeritz, A.
- The untold story of multi-mode (online and mail) consumer panels; 2014; McCutcheon, A. L., Rao, K., Kaminska, O.
- Online panels and validity; 2014; Groenlund, K., Strandberg, K.
- Assessing representativeness of a probability-based online panel in Germany; 2014; Struminskaya, B., Kaczmirek, L., Schaurer, I., Bandilla, W.
- A critical review of studies investigating the quality of data obtained with online panels based on...; 2014; Callegaro, M., Villar, A., Yeager, D. S., Krosnick, J. A.
- Online panel research: History, concepts, applications and a look at the future; 2014; Callegaro, M., Baker, R., Bethlehem, J., Goeritz, A., Krosnick, J. A., Lavrakas, P. J.
- Motives for joining nonprobability online panels and their association with survey participation behavior...; 2014; Keusch, F., Batinic, B., Mayerhofer, W.
- Targeting the bias – the impact of mass media attention on sample composition and representativeness...; 2014; Steinmetz, S., Oez, F., Tijdens, K. G.
- Exploring selection biases for developing countries - is the web a promising tool for data collection...; 2014; Tijdens, K. G., Steinmetz, S.
- The quality of ego-centered social network data in web surveys: experiments with a visual elicitation...; 2014; Marcin, B., Matzat, U., Snijders, C.
- Switching the polarity of answer options within the questionnaire and using various numbering schemes...; 2014; Struminskaya, B., Schaurer, I., Bosnjak, M.
- Measuring the very long, fuzzy tail in the occupational distribution in web-surveys; 2014; Tijdens, K. G.
- Interest Bias – An Extreme Form of Self-Selection?; 2014; Cape, P. J., Reichert, K.
- Online Qualitative Research – Personality Matters ; 2014; Tress, F., Doessel, C.
- Recent Books and Journals in Public Opinion, Survey Methods, and Survey Statistics; 2014; Callegaro, M.
- Does Gamification Work? - A Literature Review of Empirical Studies on Gamification ; 2014; Hamari, J., Koivisto, J., Sarsa, H.
- The Use of Paradata to Predict Future Cooperation in a Panel Study; 2014; Funke, F., Goeritz, A.
- Pret met panels [Fun online]; 2013; Roberts, A., de Leeuw, E. D., Hox, J., Klausch, L. T., de Jongh, A.
- The Short-term Campaign Panel of the German Longitudinal Election Study 2009. Design, Implementation...; 2013; Steinbrecher, M., Rossmann, J.
- The Future of Social Media, Sociality, and Survey Research; 2013; Hill, C., Dever, J. A.
- Second Life as a Survey Lab: Exploring the Randomized Response Technique in a Virtual Setting; 2013; Richards, A., Dean, E.
- Virtual Cognitive Interviewing Using Skype and Second Life; 2013; Dean, E., Head, B., Swicegood, J. E.
- Social Media, Sociality, and Survey Research; 2013; Hill, C., Dean, E., Murphy, J.
- Investigation of background acoustical effect on online surveys: A case study of a farmers' market...; 2013; Tang, Xi.
- Should the third reminder be sent? The role of survey response timing on web survey results; 2013; Rao, K., Pennington, J.
- Web panel surveys – can they be designed and used in a scientifically sound way?; 2013; Svensson, J.
- Using an Item Response Theory Approach to Measure Survey Mode of Administration Effects: Analysis of...; 2013; Mariano, L. T., Elliott, M. N.